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Anchor Tests 


1. Used for Non-Equivalent Anchor Test (NEAT) 
design 

o to link test forms to be equated 
o by measuring differences between populations 
o when equivalent groups are not available; 

2. Common practice building anchors 

o Miniature version of the total test 

o Representative of content and statistical specification 
of the full test 
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Anchor Tests 


• Statistical representativeness 

• Equal mean and equal standard deviation of item difficulties 

• Cover difficulty range of the full test 

• Sinharay/Holland Study (2007) 

• Suggests to relax some statistical requirements for external 
anchors 

• Compared “mini” (same SD) and “midi” (reduced SD) anchor 
tests 

• Recommendation: SD can be reduced for external anchors 
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• Link multiple forms to mimic a multi-year testing program with 
multiple forms administered in a longer time spam - not only 
two forms 

• Evaluation of scale drift across the chain using “mini” vs. “midi” 
anchor concept proposed by Sinharay/Holland (2007) 

• Scale drift across equating designs (NEAT vs. RG) within each type 
of anchor 

• Scale drift across anchor types within equating designs 

• Current study establishes a baseline for later comparisons 
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Design and Data Simulation 


• Mathematics Test of a College admission exam was used to obtain initial 
parameter estimates 

• Main test: 54 items 

• External anchor tests: 20 items 

• All items are dichotomous 

• 2PL IRT model (w/ Parscale) 

• Two series of six tests are generated with external anchors 

• Two anchor tests are embedded into each test form to facilitate common 
item equating 
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Equating Design 


* 



6 


CollegeBoard 

inspiring minds' 








Simulation 


i. 


2 . 


3 - 


4 - 


Built 2 series of tests: a “mini” (equal SD) chain and a “midi” 
(0.5 SD) chain 

Anchor tests that link consecutive forms not only needed to 
be parallel but their item parameters had to be the same as 
they represent the same item set 

Means of the item difficulties for all six main forms needed to 
be the same as the old main test form to create equivalent 
groups. 

Item parameters needed to be fixed across the two scenarios 
to be able to compare the drift across the two anchor types. 
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Data Simulation 


• 100 replications for each test (600 across the chain) 

• Baseline condition: 

o Equal form difficulty across six test forms 
o No difference in ability from year-to-year 
o Fixed test sizes (54, 20, 20) 

o 4,000 simulees for each form 

• Simulation condition: anchor type (“mini” vs. “midi”) 
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Simulation: item difficulty statistics 



Main Test -0.59 0.89 -0.59 1.00 -0.59 1.21 -0.59 1.07 -0.59 1.10 -0.59 0.96 

Anchor 1 - - -0.59 1.10 -0.59 -0.59 0.91 -0.59 -0.59 0.87 

Anchor 2 -0.59 1.10 -0.59 -0.59 0.91 -0.59 -0.59 0.87 



Main Test -0.59 0.89 -0.59 1.00 -0.59 1.21 -0.59 1.07 -0.59 1.10 -0.59 0.96 

Anchor 1 0.59 0.58 -0.59 -0.59 0.64 -0.59 -0.59 0.42 

Anchor 2 -0.59 0.58 -0.59 -0.59 0.64 -0.59 -0.59 0.42 
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RMSE (a) 


Simulation: parameter recovery for “a” - equal SD 
(mini) 
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Simulation: parameter recovery for “b” - equal SD 
(mini) 
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Location (b) 
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RMSE (a) 


Simulation: parameter recovery for “a” - 
reduced SD (midi) 
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RMSE (b) 


Simulation: parameter recovery for “b” - 
reduced SD (midi) 
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Location (b) 
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Evaluation Criteria 


RMSD = 


MAD 


MSD 
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Results 
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Results 


Form 0 

Form 1 

Form 2 

Form 3 

Form 4 

Form5 


Mini Anchor Condition 


Main - Anchor 1 

0.88 

0.86 

0.89 

0.87 

0.88 

0.88 

Main - Anchor 2 

0.87 

0.89 

0.87 

0.89 

0.87 

0.89 

Anchor 1 - Anchor 2 
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Midi Anchor Condition 
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Results 


Form 1 

Form 2 

Form 3 

Form 4 

Form5 

Mini Anchor Condition 

RMSD 

0.36 

0.44 
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0.53 
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MAD 
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0.36 
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Midi Anchor Condition 
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0.36 

0.46 

0.48 

0.53 

0.56 

MAD 

0.27 

0.34 
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Difference in Raw Score Equivalents 


Results: NEAT vs. RG for “mini” chain 
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Difference in Raw Score Equivalents 


Results: NEAT vs. RG for “midi” chain 
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Form 1 

Form 2 

Form 3 

Form 4 

Form5 

RG Equating 

RMSD 
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Difference in Raw Score Equivalents 


Results: RG across chains (mini vs. “midi”) 
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Difference in Raw Score Equivalents 


Results: NEAT across chains (mini vs. “midi”) 
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Conclusions & Future Research 


• Results support Sinharay/Holland recommendation 

• SD can be reduced when building external anchor tests 

• Continuing research efforts: 

• Various ability levels and form difficulties; 

• Total/anchor length ratios; 

• Small sample equating; 

• Correlations between anchor and total test; 

• Effect on IRT equating methods; 

• Real data applications 
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